Systems and methods for audio adaptation of content items to endpoint media devices

ABSTRACT

Methods, systems, and non-transitory, machine-readable media are disclosed for audio adaption of content items to device operations of an endpoint media device. First observation data corresponding to media device operations associated with a first media device and mapped to first content items may be processed. A first content composite including an adaptable content item may be received. The first content composite may be adapted with a first audio segment. Based on the first observation data, the first audio segment may be selected. The first content composite may be configured with the first audio segment so that the adapted first content composite plays the first audio segment when the adapted first content composite is presented. The adapted first content composite may be output for presentation, where the first endpoint media device or the second endpoint media device performs at least one operation relating to the adapted first content composite.

This disclosure generally relates to content delivery, and moreparticularly to systems and methods for adapting content items toendpoint media devices.

BACKGROUND

The evolution of wireless networks and content distribution platformsmay present a number of challenges and problems for content deliverywithin packet streams per Internet Protocol (IP), the ATSC (AdvancedTelevision Systems Committee) 3.0 standard, or a like protocol. Suchmodes of content delivery may not be linear, in contrast to traditionaltelevision delivery, for example. And, still, there are needs forintegrating more deeply with endpoint devices in broadcast and otherdistribution environments. Content viewers are in need of better viewerexperiences and more tailored service offerings. Thus, there is a needfor systems and methods that address such problems. This and other needsare addressed by the present disclosure.

BRIEF SUMMARY

Certain embodiments of the present disclosure relate generally tocontent delivery, and more particularly to systems and methods foradapting content items to endpoint media devices.

In various aspects, a method, a system, and one or more non-transitory,machine-readable media are disclosed for audio adaption of content itemsto device operations of an endpoint media device, and may include one ora combination of the following operations. A first set of observationdata corresponding to indications of detected media device operationsassociated with a first media device and mapped to a first set ofcontent items may be processed. A first content composite including anadaptable content item corresponding to a set of one or more audioand/or video packets corresponding to audio and/or video content may bereceived, the first content composite from a content provider system.The first content composite may be adapted with a first audio segment.The adapting may include selecting, based at least in part on the firstset of observation data, the first audio segment from a plurality ofaudio segments. The adapting may include configuring the first contentcomposite with the first audio segment so that the adapted first contentcomposite plays the first audio segment for at least part of a firstpresentation time when the adapted first content composite is presented.The adapted first content composite may be output for presentation,where, consequent to providing the adapted first content composite tothe first endpoint media device or a second endpoint media device, thefirst endpoint media device or the second endpoint media device performsat least one operation relating to the adapted first content composite.At least one communication received from the first endpoint mediadevice, the second endpoint media device, or a remote system, the firstendpoint media device, and the second endpoint media device may beprocessed. The at least one communication may be indicative of the atleast one operation relating to the adapted first content composite. Asecond content composite may be adapted with a second audio segment. Theadapting may include selecting, based at least in part on the at leastone communication indicative of the at least one operation relating tothe adapted first content composite, the second audio segment. Theadapting may include configuring the second content composite with thesecond audio segment so that the adapted second content composite playsthe second audio segment for at least part of a second presentation timewhen the adapted second content composite is presented. The adaptedsecond content composite may be output for presentation, where,consequent to providing the adapted second content composite to thefirst endpoint media device or the second endpoint media device, thefirst endpoint media device or the second endpoint media device performsat least one operation relating to the adapted second content composite.

In various embodiments, the plurality of audio segments may be receivedfrom the content provider system. In various embodiments, the pluralityof audio segments may be received with the first content composite. Invarious embodiments, the first content composite may be adapted with thefirst audio segment as a function of time. In various embodiments, thefirst set of observation data may include a viewer metric as a functionof time, and the first content composite may be adapted with the firstaudio segment as a function of the viewer metric. In variousembodiments, the first set of observation data may include a currentgeolocation of the first endpoint media device, and the first contentcomposite may be adapted with the first audio segment as a function ofthe current geolocation. In various embodiments, the at least oneoperation relating to the adapted first content composite may correspondto storing a code corresponding to the first audio segment by the firstendpoint media device and/or the second endpoint media device.

Further areas of applicability of the present disclosure will becomeapparent from the detailed description provided hereinafter. It shouldbe understood that the detailed description and specific examples, whileindicating various embodiments, are intended for purposes ofillustration only and are not intended to necessarily limit the scope ofthe disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

A further understanding of the nature and advantages of variousembodiments may be realized by reference to the following figures. Inthe appended figures, similar components or features may have the samereference label. Further, various components of the same type may bedistinguished by following the reference label by a dash and a secondlabel that distinguishes among the similar components. If only the firstreference label is used in the specification, the description isapplicable to any one of the similar components having the same firstreference label irrespective of the second reference label.

FIG. 1 illustrates a content distribution system, in accordance withdisclosed embodiments of the present disclosure.

FIG. 2 illustrates a diagram of an adaptive content composite system, inaccordance with disclosed embodiments of the present disclosure.

FIG. 3 illustrates a composite build engine, in accordance withdisclosed embodiments of the present disclosure.

FIG. 4 illustrates certain aspects of subsystem data flow, in accordancewith disclosed embodiments of the present disclosure.

DETAILED DESCRIPTION

The ensuing description provides preferred exemplary embodiment(s) only,and is not intended to limit the scope, applicability or configurationof the disclosure. Rather, the ensuing description of the preferredexemplary embodiment(s) will provide those skilled in the art with anenabling description for implementing a preferred exemplary embodimentof the disclosure. It should be understood that various changes may bemade in the function and arrangement of elements without departing fromthe spirit and scope of the disclosure as set forth in the appendedclaims.

Various embodiments will now be discussed in greater detail withreference to the accompanying figures, beginning with FIG. 1. FIG. 1illustrates a content distribution system 100, in accordance withdisclosed embodiments of the present disclosure. For brevity, system 100is depicted in a simplified and conceptual form, and may generallyinclude more or fewer systems, devices, networks, and/or othercomponents as desired. Further, the number and types of features orelements incorporated within the system 100 may or may not beimplementation-specific, and at least some of the aspects of the system100 may be similar to a cable television distribution system, an IPTV(Internet Protocol Television) content distribution system, and/oranother type of media or content distribution system.

In general, the system 100 may include a plurality of networks 120 thatcan be used for bi-directional communication paths for data transferbetween components of system 100. Disclosed embodiments may transmit andreceive data, including video content, via the networks 120 using anysuitable protocol(s), as is disclosed in co-pending U.S. patentapplication Ser. No. 16/383,300, filed Apr. 12, 2019, the disclosure ofwhich application is incorporated by reference herein in its entiretyfor all purposes, as if fully set forth herein. The system 100 mayinclude one or more content provider systems 102, satellite uplinks 104,a plurality of orbiting (e.g., geosynchronous) satellites 106, satellitereceivers 108, one or more other data source systems 103, one or morecontent sources 112, and media devices including adaptive routers andvarious endpoint devices 116 (e.g., computing devices 116 a-d,television receivers 116 e, and/or one or more display devices 160)communicatively couplable via the networks 120, further details of whichare disclosed in co-pending U.S. patent application Ser. No. 16/655,104,filed Oct. 16, 2019, the disclosure of which application is incorporatedby reference herein in its entirety for all purposes, as if fully setforth herein.

The content provider system 102 may include one or more adaptablecontent item 176 repositories and/or particular content item 177repositories. The content provider system 102 may store adaptablecontent items 176 and/or particular content item 177 in the one or morerepositories. The one or more repositories may be implemented in variousways. For example, one or more data processing systems may storeadaptable content items. One or more relational or object-orienteddatabases, or flat files on one or more computers or networked storagedevices, may store adaptable content items. In some embodiments, acentralized system stores adaptable content items; alternatively, adistributed/cloud system, network-based system, such as beingimplemented with a peer-to-peer network, or Internet, may storeadaptable content items.

Adaptable content items 176 and/or particularized content items 177 maycorrespond to any one or combination of raw data, unstructured data,structured data, information, and/or content which may include mediacontent, text, documents, files, instructions, code, executable files,images, video, audio segments and/or tracks, audio video, and/or anyother suitable content suitable for embodiments of the presentdisclosure. For example, the content items 176 may be adaptable contentitems that may correspond to visual and/or audiovisual content withgraphical and/or audio components particularized to certain types ofproducts and/or services. In some embodiments, the content items 176 maycorrespond to commercials to be presented during commercial breaks oftelevision programming, such as televised events. In some instances, thecontent items 176 may be sourced by one or more of the service providersystems 103.

Some embodiments may include analyzing audio and/or video of theitems/composite in order to facilitate the adapting thereof. In someembodiments, the adaptable content items 176 may correspond to videoand/or audio video file structures with one or more transition points,hooks, frames, windows, and/or the like for merging with one or moreparticularized content items, content items 177, particularized tocertain products, services, geolocations, and/or languages. In variousembodiments, the adaptable content items 176 may be adapted withparticularized content items 177 to create and/or adapt electroniccontent composites 180 particularized to certain types of products,services, geolocations, contact information, hyperlinks, viewers, times,and/or languages. In some embodiments, the content composites 180 may betransmitted to endpoint devices 116 with multiple options of contentitems 177 for the endpoint device 116 to select and customize thecontent composite 180 for presentation. In various embodiments,adaptable content items 176 and multiple options of content items 177may be transmitted to endpoint devices 116 separately and/or in anintegrated manner with the content items 176 and content items 177integrated within composites 180. In various embodiments, content items176 and content composites 180 may be adapted with particularizedcontent items 177 to adapt electronic content composites 180 so that thecontent composites 180 are particularized. As disclosed herein,adaptation of content items 176 and content composites 180 may be afunction of a number of factors and may be based at least in part onobservation data (which may, in some embodiments, be received viabackchannel notifications) that is indicative of detected media deviceoperations associated with particular media devices 116 and mapped toparticular content items 176, 177, and/or 202. The factors may includecontextual factors, such as bandwidth, endpoint metrics, programmingcontent being presented, endpoint inferences, user preferences, time ofday, and/or the like.

For example, as disclosed herein, the adaptable content items 176 may bemerged, blended, joined, overlaid, customized, and/or the like in anysuitable manner with other content items 177, selected based in part onobservation data indicative of detected media device operations in someembodiments, in order to create electronic content composites 180particularized to certain types of products, services, geolocations,contact information, hyperlinks, viewers, times, and/or languages. Invarious embodiments, as further disclosed herein, the adaptable contentitems 176 and/or the other content items 177 may be formatted, rescaled,cropped, image characteristic (e.g., color, brightness, transparency,opaqueness, contrast, etc.) adjusted, and/or otherwise prepared tofacilitate the merging, blending, joining, overlaying, customizing,and/or the like and presentation by endpoint media devices 116 asdisclosed herein. As another example, as disclosed herein, adaptablecontent items 176 and/or electronic content composites 180 may beadapted with content items 177 corresponding to audio segmentsparticularized to certain types of products, services, geolocations,contact information, hyperlinks, viewers, times, and/or languages. Forinstance, the adapting may include selecting, based at least in part ona set of observation data indicative of detected media deviceoperations, an audio segment from a plurality of audio segments andconfiguring an adaptable content item 176 and/or electronic contentcomposite 180 with the selected audio segment so that the resultingcontent composite 180 plays the selected audio segment for at least partof a first presentation time when the content composite 180 ispresented. In various embodiments, the configuring may include adding anaudio segment to the adaptable content item 176 and/or electroniccontent composite 180 and/or replacing an audio segment thereof. Invarious embodiments, replacing an audio segment may include detaching(e.g., dereferencing, unselecting, or otherwise disassociating an audiofile from a play command that would cause the audio file to be playedalong with the playing of the video file of the item/composite),cutting, and/or deleting a previously attached audio track/file from thevideo file and attaching a different audio track/file to the video file.In various embodiments, the audio segment may be new to, or selectedfrom, the audio segment to the adaptable content item 176 and/orelectronic content composite 180. For example, an electronic contentcomposite 180 may include a plurality of audio segments from which oneor more audio segments may be selected to play with the video portion ofthe composite 180. In various embodiments, the audio segment may beconfigured to play as an entire audio track commensurate with thepresentation time of the video portion and/or to play for only part ofthe video timeline. In some embodiments, the configuring may includemanipulating the file to mix multiple audio segments, to adjust thevolumes of the audio segments, to combine audio segments at variousvolume levels, to fade into and/or out of audio segments, to trim thelengths of the audio segments, to combine audio segments based at leastin part on matching waveforms, audio elements (e.g., beats, melodic,harmonic), and/or audio characteristics (e.g., tempo, volume, duration)of the audio segments, to adjust the positions the audio segments toparticular positions with respect to the video timelines, and/or thelike.

In various embodiments, the particularized content items 177 maycorrespond to content that is particularized to certain types ofproducts and/or services and that, in various embodiments, may besourced by one or combination of the system 102, one or more of theservice provider systems 103, and/or the endpoint media device 116(e.g., in instances where personalization data that may form at leastpart of a particularized content item 177). In various embodiments, theservice provider systems 103 may correspond to one or more sources ofdata and/or services corresponding to the adaptable content items 176and/or the particularized content items 177, and particularized contentitems 177 may correspond to the specific data and/or services sourced bya specific service provider system 103. For example, the data maycorrespond to one or more audio segments, a particular ecoupon, matrixcode such as a QR code, a promo code, trailer, movie content, and/or thelike selected for presentation as a function of a number of factors. Thefactors may include contextual factors, such as bandwidth, endpointmetrics, programming content being presented, endpoint inferences, userpreferences, time of day, and/or the like. Such data may beparticularized to a viewer category (e.g., child with a preference for aparticular language and dialect). In some embodiments, such data maycorrespond to one or more audio segments that represent an ecoupon or apromo code, and that may be configured to invoke, awaken, open, orotherwise activate an application on an endpoint device 116 and promptthe saving of the ecoupon or promo code in the endpoint device 116. Insome embodiments, such prompting may be performed at least in part byaudio cues and commands directed to a virtual assistant of a nearbyendpoint device 116 (e.g., a smart speaker). In some embodiments,particularized content items 177 may include personalized content thatis particular to one or more viewers. In some embodiments, thepersonalized content may be separate from the particularized contentitems 177. As disclosed above, the adaptable content items 176 and/orthe content items 177 may correspond to any one or combination of rawdata, unstructured data, structured data, information, and/or contentwhich may include media content, text, documents, files, instructions,code, executable files, images, video, audio, audio video, and/or anyother suitable content suitable for embodiments of the presentdisclosure.

In various embodiments, content items 176 and/or 177 may be activelygathered and/or pulled from one or more data sources 112, for example,by accessing a repository and/or by “crawling” various repositories.Additionally or alternatively, the content provider system 102 and/orthe subsystem(s) 111 may monitor for updates from one or a combinationof the content source systems 112. Content items 176 and/or 177 pulledand/or pushed from the one or more data sources 112 may be transformed,and the transformed content items 176 and/or 177 and/or other datagenerated based thereon may be made available by the content providersystem 102 and/or the subsystem(s) 111 for use by the subsystem(s) 111in conjunction with content composites 180. Additional details regardingthe transfer of content items 176, 177, and composites 180 are providedin the incorporated by reference applications. The content items 176and/or 177 may be pushed and/or pulled consequent to gathering andanalyzing observation data disclosed herein to identify content metricsand/or contextual factors for correlation to and identification ofcorresponding content items 176 and/or 177. For example, the contentitems 176 and/or 177 may be identified as corresponding to one or acombination of the content metrics, intra-content metrics, and/or thecontextual factors that may include bandwidth, endpoint metrics,programming content being presented, endpoint inferences, userpreferences, time of day, and/or the like, as disclosed herein.Accordingly, the content items 176 and/or 177, as well as thecorresponding composites 180, may be particularized to one or acombination of such contextual factors and provisioned for particularviewers.

The content provider system 102 may include a content compositesubsystem 111 in whole or in part. In some embodiments, additionally oralternatively, one or more of the endpoint media devices 116 may includea content composite subsystem 111 whole or in part. In some embodiments,additionally or alternatively, one or more service provider systems 103may include a content composite subsystem 111 in whole or in part. Thecontent composite subsystem 111 may be configured to facilitate variouscontent adaptation features in accordance with various embodimentsdisclosed herein.

The system 102 with the content composite subsystem 111 may beconfigured to perform one or more methods for facilitating adaptivecontent items for delivery in a packet stream, such as methods forcontainerizing and adapting content items disclosed herein. For example,the system 102 with the content composite subsystem 111 may beconfigured to perform a method for audio adaption of content items todevice operations of an endpoint media device disclosed herein. Invarious embodiments, part or all of the method may be performed while anendpoint media device 116 is receiving programming content and/or isoutputting programming content for display. In various embodiments, atleast part of the method may be performed in advance of the programmingcontent being received by the endpoint media device 116 and, thus, maybe performed before the programming content is broadcast to endpointmedia devices 116 and/or before the programming content output by anendpoint media device 116 for display. In various embodiments, one ormore media devices (e.g., the devices 116 a-e and/or the system 102) mayperform all or part of the method, with a single media device ormultiple media devices performing the method.

One or more content composites 180 may be created and adapted tofacilitate delivery of the one or more composites 180 and display of oneor more content items 176, as modified with particular items 177, inconjunction with the televised event or other type of audiovisualcontent (movies, shows, etc.), e.g., so that the composites 180correspond to the content (e.g., are directed to the same type ofviewer, such as a child with a preference for a particular language anddialect). One or more indicators of one or more state changes and one ormore content metrics with respect to the content (e.g., event that istelevised, show, etc.) may be detected by the subsystem 111. The one ormore state changes may include upcoming state changes, such ascommercial breaks upcoming within a time threshold (e.g., a number ofseconds and/or minutes). In some embodiments, the one or more contentcomposites 180 may be created and/or delivered consequent to thedetecting the one or more state changes. In some embodiments, the one ormore content items 176 may be adapted with one or more particular items177 consequent to the detecting the one or more state changes. The oneor more content metrics with respect to the content may includeintra-content metrics corresponding to indicia or specifications oflanguages and dialects of dialogues and/or text within the contentand/or selected closed captioning. The one or more content metrics withrespect to the content may include indicia or specifications of types ofcontent (e.g., ratings for different audience designations, contentdesignations, etc.).

The creating the content composite 180 and/or the adaptation of the oneor more content items 176 with one or more particular items 177 may be afunction of a current geolocation of the endpoint media device 116, witha set of rules mapped to the current geolocation and specifyinggeo-specific criteria for creating/adapting content composites 180,selecting content items 176 and particularized content items 177,adapting the content items 176 with particularized content items 177,and provisioning the content items 176 and particularized content items177. The current geolocation of the media device may be determined at atime when the content is being output for display (e.g., immediatelyprior to when the composite 180 is to be output for display). Indicia orspecifications of the current geolocation may be included in andgathered with the observation data. With the content composite 180created and, in some instances, the one or more content items 176adapted, the one or more content items 176 corresponding to theprogramming content may be output for display, e.g., during thecommercial break, where the content items 176 and particularized contentitems 177 are selected based at least in part on location metadatamapped to the content items 176 and particularized content items 177specifying location indicia for the content items 176 and particularizedcontent items 177. Some sets of rules may specify a threshold distance,and the content items 176 and particularized content items 177 may beselected when a distance between the current geolocation of the endpointmedia device 116 and the location indicia for the content items 176 andparticularized content items 177 satisfies the distance threshold. Somesets of rules may specify that only certain types of particularizedcontent items 177 may be selected when the distance threshold is notsatisfied (e.g., promo codes but not other types).

Disclosed embodiments may confirm and verify that a content item 176 (insome instances, as modified with one or more particular content items177, such as one or more audio segments, matrix code such as a QR code,promo code, particularization data, and/or the like) was transmitted toan endpoint media device 116. Disclosed embodiments may confirm andverify device operations indicating that the modified or unmodifiedcontent item 176 was presented with a display device at the endpointmedia device 116 (which display device may be integrated with theendpoint media device 116 or otherwise communicatively coupled to theendpoint media device 116). Disclosed embodiments may confirm and verifythat the modified or unmodified content item 176 was shown to anintended end user if and when utilizing encryption and/or an encryptionflag. Disclosed embodiments may confirm and verify device operationsindicating to what extent the modified or unmodified content item 176was presented (in full or to a lesser extent). Disclosed embodiments mayconfirm and verify detection of one or more operations executedconsequent to the presentation (e.g., activating an application on anendpoint device 116 and prompting the saving of an ecoupon or promo codein the endpoint device 116, automated transcribing an audio segmentportion to facilitate the saving of the item in the endpoint device 116,scanning a matrix code, loading the item and/or another particularizedcontent item 177 into mobile app such as a digital wallet of theendpoint media device 116, utilizing the particularized content items177 such as to order a product and/or service, etc.) and metrics of theoperations (e.g., time of execution). In various embodiments, one or acombination of the above may be confirmed and verified based at least inpart on observation data indicative of detected media device operations.

Such observation data may be gathered by way of return channelnotifications from the endpoint device 116 to the system 102, which maycontribute to the observation data 229 and feedback loop featuresdisclosed further herein. In addition or in alternative to suchcommunications from endpoint devices 116, the system 102 may receivepushed and/or pulled notifications from one or more service providersystems 103 that indicate when particularized content items 177 havebeen used by the endpoint devices 116 with respect to the systems 103(e.g., to a system 103, uploading at least a portion of a content item177; communicating at least a portion of a content item 177 such ascode, accessing using a URL of a content item 177; transferring,presenting, or otherwise providing an image capture of a content item177; and/or the like). Such notifications may also contribute to theobservation data 229 and feedback loop features.

FIG. 2 illustrates a functional diagram of an adaptive content compositesystem 200, in accordance with disclosed embodiments of the presentdisclosure. In various embodiments, the content composite system 200 maybe included in whole or in part in the content provider system 102and/or an endpoint media device 116. In some embodiments, the contentcomposite system 200 may be separate from, and provide content to, thecontent provider system 102. In some embodiments, the content compositesystem 200 may be included in the end-user system and may be included inthe television receiver 116 e and/or one or more of the computingdevices 116. In some embodiments, various features of the contentcomposite system 200 may be distributed between the television receiver116 e and upstream of the television receiver 116 e. Likewise, in someembodiments, various features of the content composite system 200 may bedistributed between one or more of the computing devices 116 andupstream of the one or more computing devices 116. While not allcomponents of the adaptive content composite system 200 are shown, thesystem 200 may include one or a combination of such components.

As depicted, the content composite system 200 may include a contentcomposite subsystem 111. The content composite subsystem 111 may includeor otherwise correspond to an audiovisual control engine that, as withdisclosed embodiments of the other engines, may include instructionsretained in processor-readable media and to be executed by one or moreprocessors. The content composite subsystem 111 may be communicativelycoupled with interface components and communication channels (e.g., ofthe television receiver 116 e and/or the computing device 116, which maytake various forms in various embodiments as disclosed herein)configured to receive programming content 202, which may correspond totelevised sporting events, movies, television programs, portionsthereof, etc. In various embodiments, the programming content 202 mayinclude audiovisual content broadcast and/or otherwise transmitted bythe content provider system 102 and/or one or more other serviceproviders 103. The programming content 202 may include variouscomponents, including without limitation, one or more video tracks,audio tracks, audio video tracks, metadata tracks, close captioninginformation, and/or the like. In some embodiments, the content compositesystem 200 may retain received programming content 202 in contentstorage 222. The content storage 222 may include any suitable form ofstorage media, such as any suitable form disclosed herein.

The content composite subsystem 111 may be further configured to receiveadaptable content items 176 and particularized content items 177.Additionally, the composite subsystem 111 may be further configured toreceive content composites 180 in some embodiments. The contentcomposite subsystem 111 may include a harvesting engine 236 configuredto aggregate adaptable content items 176, particularized content items177, content composites 180, and/or programming content 202 in order tofacilitate content splicing features disclosed herein. The contentcomposite subsystem 111 may include a matching engine 238, which, invarious embodiments, may be configured to analyze, classify, categorize,characterize, tag, and/or annotate adaptable content items 176,particularized content items 177, content composites 180, and/orprogramming content 202.

The content composite subsystem 111 may include a content splicingengine 242. In some embodiments, the content splicing engine 242 mayinclude a multiplexer. In various embodiments, the multiplexer maycreate a digital stream of data packets containing the video, audio,and, in some embodiments, the metadata to output the programming content202, adaptable content items 176, and/or the composites 180 createdand/or adapted with selected particularized content items 177. Invarious embodiments, the content splicing engine 242 may be implementedat the receiver 116 e, the device 116, and/or the service providersystem 102. In some embodiments where the content splicing engine 242 isimplemented at the service provider system 102, the multiplexed datastream may be transmitted via the one or more networks 124 forprovisioning to computing devices 116 or via a particular transponderstream via a transponder of a satellite for provisioning to receivers116 e as further detailed in the incorporated applications.

The content splicing engine 242 may, in various embodiments, correspondto a processor, such as a main processor, a core processor, digitalsignal processor, and/or like. In some embodiments, the content splicingengine 242 may perform at least part of the adapting and configuring ofcontent items 176 with content items 177. The content splicing engine242 or another component of the subsystem 111 may select one or moreaudio segments 177, for example, based at least in part on a set ofobservation data indicative of detected media device operations. Thecontent splicing engine 242 may configure an adaptable content item 176with the selected one or more audio segments 177 to play the selectedaudio segment for at least part of a first presentation time when thecorresponding content composite 180 is presented. In variousembodiments, the content splicing engine 242 may add one or more audiosegments 177 to an adaptable content item 176 and/or replace an audiosegment of the adaptable content item 176. The content splicing engine242 may identify one or more portions of an adaptable content item 176to be adapted based at least in part on one or more content items 177.In various embodiments, the content splicing engine 242 may identify byaudio and/or video segment/file analysis and/or analysis of tag datathat defines one or more areas within the adaptable content item 176that correspond to certain portions for adaptation. As disclosed herein,such analyses and/or tag data may define an audio segment of the contentitem 176 for replacing at least partially. In some embodiments, an audiosegment of the content item 176 may not be replaced, where the contentitem 176 has no audio and/or one or more audio content items 177 areadded to the content item 176, which may include layering on and/ormixing with one or more audio content items 177 of the content item 176.In various embodiments, the content splicing engine 242 may replace anaudio segment by detaching (e.g., dereferencing, unselecting, orotherwise disassociating an audio file from a play command that wouldcause the audio file to be played along with the playing of the videofile of the item/composite), cutting, and/or deleting a previouslyattached audio track/file from the video file and attaching one or moredifferent audio tracks/files 177 to the video file. In variousembodiments, selected audio segment may be configured to play as anentire audio track commensurate with the presentation time of the videoportion of the content item 176 and/or to play for only part of thevideo timeline. In various embodiments, the content splicing engine 242may manipulate the file to mix multiple audio segments, to adjust thevolumes of the audio segments, to combine audio segments at variousvolume levels, to fade into and/or out of audio segments, to trim thelengths of the audio segments, to combine audio segments based at leastin part on matching waveforms, audio elements (e.g., beats, melodic,harmonic), and/or audio characteristics (e.g., tempo, volume, duration)of the audio segments, to adjust the positions the audio segments toparticular positions with respect to the video timelines, and/or thelike. The content splicing engine 242 may, in some embodiments,configure the content item 176 with a set of audio segments 177 fromwhich one or more of the audio segments 177 may be selected to play withthe content item 176 as part of the composite 180.

In some embodiments, the harvesting engine 236 may be configured toreceive, pull, process, buffer, organize, rank, and/or store adaptablecontent items 176, particularized content items 177, content composites180, and/or programming content 202 as a function of one or acombination of the content metrics and/or the contextual factors. Forexample, the adaptable content items 176, particularized content items177, content composites 180, and/or programming content 202 may becategorized and ranked according to identification as corresponding toone or a combination of the content metrics and/or contextual factorsincluding bandwidth, endpoint metrics, programming content beingpresented, endpoint inferences, user preferences, time of day, and/orthe like, as disclosed herein. Consequently, further features ofadaptation and provisioning disclosed herein may then be provided basedat least in part on the categorizations and rankings.

In various embodiments, the content provider system 102, the televisionreceiver 116 e, and/or the computing device 116 may include one or moreapplications to facilitate the subsystem 111 analyzing and consolidatingdata feeds and/or event updates received from various data sources 112.As an example, data feeds and/or event updates may include, but are notlimited to, updates (real-time and/or otherwise) and/or continuous datastreams received from one or more data sources 112, which may includereal-time events related to promotions, redeemable content items,special offerings, discounts, sports event information, Twitter® feeds,Instagram® posts, Facebook® updates, Amazon® updates, and/or the like.

As disclosed above, the adaptable content items 176 and composites 180may be particularized. In some embodiments, the adaptable content items176 and composites 180 may correspond to commercials to be presentedduring commercial breaks of the programming content 202. In variousembodiments, the content items 176 and composites 180 may includeaudiovisual content broadcast and/or otherwise transmitted by thecontent provider system 102. In some embodiments, adaptable contentitems 176 and/or composites 180 may be pushed by the content providersystem 102 to the subsystem 111. In addition or in alternative,adaptable content items 176 and/or composites 180 may be pulled by thesubsystem 111 (e.g., by the harvesting engine 236) from the contentprovider system 102.

In various embodiments, sets of one or more adaptable content items 176sets of one or more content items 177, and/or sets of one or morecomposites 180 may be transmitted to the subsystem 111 in batches. Forexample, sets may be transmitted to the subsystem 111 on a periodic orotherwise specified basis. In some embodiments, the subsystem 111 maystore the sets locally and, subsequently select one or more of theadaptable content items 176 and/or composites 180, as well as one ormore content items 177 with which to adapt the one or more adaptablecontent items 176 and/or composites 180 when needed for presentationduring an upcoming break in the programming content 202 and/or whenneeded for presentation during the programming content 202 based atleast in part on the subsystem 111 processing observation dataindicative of detected media device operations and using the observationdata as one or more bases for the selection and adaptation.

In various embodiments, sets of one or more adaptable content items 176sets of one or more content items 177, and/or sets of one or morecomposites 180 may be transmitted to the subsystem 111 on an as-neededbasis when the subsystem 111 is receiving programming content 202corresponding to a certain type (e.g., a televised sporting event), isspecified to receive such programming content 202, is predicted fromobservation data to receive programming content 202 based at least inpart on a detected viewing pattern of past viewing of previousprogramming content 202 (e.g., of a certain type of event, at certaintimes, on certain days, etc.), and/or is predicted from observation datato receive programming content 202 based at least in part on a detectedpattern of past viewer responses to content composites 180 for previousprogramming content 202 of that type. Additionally or alternatively, insome embodiments, sets of one or more adaptable content items 176 and/orsets of one or more content items 177 may be selected based at least inpart on the observation data indicative of media device operations(e.g., by the service provider system 102 and/or the subsystem 111) astailored for particular event viewing habits, ordering patterns, andinferred interests of viewers.

In various embodiments, sets of one or more adaptable content items 176sets of one or more content items 177, and/or sets of one or morecomposites 180 may be selected (e.g., by the service provider system102) for particular time periods and may be transmitted to the subsystem111 with an assignment (e.g., by way of tag data or other metadata) forthe designated time period. Additionally or alternatively, in someembodiments, sets of one or more adaptable content items 176 sets of oneor more content items 177, and/or sets of one or more composites 180 maybe selected (e.g., by the service provider system 102) for particularchannels and/or television programs and may be transmitted to thesubsystem 111 with an assignment (e.g., by way of tag data or othermetadata) for the designated channels, shows, movies, and/or televisionprograms. The transmission of the sets may be in response to thesubsystem 111 pulling the sets from the service provider system 102. Forexample, the subsystem 111 may pull adaptable sets of one or moreadaptable content items 176 sets of one or more content items 177,and/or sets of one or more composites 180 based at least in part on theobservation data indicative of media device operations. This may includedetecting programming content 202 currently being viewed via atelevision receiver 116 e or computing device 116, detecting programmingcontent 202 specified to be viewed or recorded, predicting programmingcontent 202 of interest to a viewer based on detected viewing patternsand/or patterns of interacting with content items 176, 177, determiningupcoming programming content 202 based on electronic programming guideinformation received, and/or the like. In a similar manner, sets of oneor more adaptable content items 176 sets of one or more content items177, and/or sets of one or more composites 180 may be pulled from, orpushed by, one or more service provider systems 103, in variousembodiments using one or more of the various methods disclosed, to thesubsystem 111 directly or indirectly (e.g., by way of the contentprovider system 102, which may then transmit the sets to the subsystem111) for particular time periods, with assignments for designatedchannels, shows, movies, television programs, etc. For example, inconjunction with the selection of sets of one or more adaptable contentitems 176, sets of one or more content items 177 that match the one ormore adaptable content items 176 may be pulled from one or more serviceprovider systems 103. In various examples, the matching may be based atleast in part on the observation data.

The subsystem 111 may use the rules 218 to adaptively control contentcomposite creation, delivery, adaptation, and interactions with suchcontent. The rules 218 may include conditions corresponding toparticulars of particularized content items 177 and may specify one ormore operations to be performed when one or more of the conditions aresatisfied. The conditions may, in some instances, be received from oneor more provider systems 103. The categorization 212 of the harvestedregulations and updated regulations may be used to create various setsof rules 218 governing the creation, provisioning, and adaptation ofparticularized content composites 180. Provisioning and/or adaptation ofcontent composites 180 may be differentiated according to show typeand/or viewer type, with time, place, and/or mannerrestrictions/specifications contingent on show type and/or viewer type,among other factors.

The various sets of rules 218 may govern the creation, provisioning, andadaptation of content items 176 and particularized content composites180 as a function of a number of factors, which may include various,differently weighted, contextual factors—such as endpoint metrics, timeof day, bandwidth, programming content being presented, endpointinferences, user preferences, and/or the like. For example, the endpointmetrics may include location, and the various sets of rules 218 maygovern the creation, provisioning, and adaptation of particularizedcontent composites 180 as a function of a location of a particularreceiver 116 e and/or device 116. As detailed in the incorporated byreference applications, location data may be captured. The location datacapture may facilitate geo-sensitive adaptive content splicing and todifferentiate which sets of rules 218 apply to a given contentprovisioning instance with respect to a program and which content items176 and/or 177 to select for adaption and provisioning of a composite180 as a function of the current device 116 location. Thus, for example,one or more particular audio segments 177 may be selected to use toadapt a content item 176 and/or composite 180 when a distance betweenthe current geolocation of the endpoint media device 116 and thelocation indicia for the one or more particular audio segments 177 iswithin a distance threshold specified by one or more sets of the rules218.

Various sets of rules 218 may provide for various types of restrictionsand/or specifications on creating, adapting, and/or provisioning contentcomposites 180. In addition or alternative to geolocationrestrictions/specifications, the various types of restrictions and/orspecifications may include restrictions on and/or specifications oftypes of content (e.g., ratings for different audience designations,content designations, etc.) for which content composites 180 may or maynot be presented and the manner in which content composites 180 may bepresented for the different types of programs and/or viewers (e.g.,adult-directed ads may not be presented during children-directed showsand/or to viewers identified as children). The various types ofrestrictions and/or specifications may further include timerestrictions, such as limits on a time of day on how the adaptation isto be done and when certain adapted content composites 180 may bepresented, limits on time in advance particular event (e.g., days,hours, etc.) and/or portion thereof ahead of which content composites180 may be presented, and the like. For example, adaptation as afunction of time of day may include selecting content items 177 directedto children when the presentation time is between 3 PM and 6 PM,selecting content items 177 directed to family when the presentationtime is between 6 PM and 10 PM, and selecting content items 177 directedto adults when the presentation time is between 10 PM and 7 AM.

However, the subsystem 111 may intelligently identify and adapt toexceptions to such temporal defaults, e.g., as a function of moreheavily weighted factors such as endpoint inferences. The subsystem 111may infer that the viewer corresponds to one or more particularcategories (e.g., a child with a preference for a particular languageand dialect) based at least in part on the programming content 202 beingviewed (e.g., cartoons in a particular Spanish dialect) and/or beingdelivered at a particular or proximate time (e.g., within a half-hourtimeframe) to the endpoint device 116 and/or based on observation datagathered via backchannel notifications regarding the programming content202 being delivered at the particular or proximate time. Say, forexample, that an endpoint inference may correspond to inferring that aviewer is a child. Based at least in part on that inference, one or moreparticular content items 177 may not be selected and/or a particularcontent item 177 may be selected. In such an example, when the inferencecorresponds to determining a child being a viewer, a particular audiotrack 177 may not be selected for presentation (e.g., that a particulartoy is available or an adult-directed audio segment) even though atemporal factor may be weighted in favor of the selection (e.g., whenthe time is after 10 PM), because the inference of the viewer typecorresponding to a child may be more heavily weighted. Further, acontent composite 180 may include particularized content items 177particularized to two or more language variants (e.g., by way of textand/or audio in 26 different languages and/or dialects). In the exampleabove of a determined viewer preference for a particular Spanishdialect, one or more audio and/or video content items 177 correspondingto the language and dialect may be selected for adaptation of thecomposite 180 and presentation to the viewer. Additionally, anyembodiment herein may conform to explicitly specified user preferences,which may override other factors such as inferences.

Additionally or alternatively, restrictions on and/or specifications ofthe manner in which content composites 180 may be adapted and presentedmay include differentiating types of devices (e.g., smart phone versuslaptop computer, laptop computer versus television receiver, etc.) whichwill display the content composites 180. Such device determinations maybe made based at least in part on the observation data in someembodiments. Consequently, certain types of adaptations may be accordedto certain types of devices. For example, audio segments 177 may be usedinstead of video items 177 to audio adapt content items 176 when thedevice is a mobile phone, and vice versa with video adaptation when thedevice is a television receiver. Certain embodiments may provide forcreation and/or adaptation of content composites 180 as a function ofbandwidth constraints. For example, the observation data and/orbackchannel communications disclosed herein may include indicia ofbandwidth abilities and/or bandwidth strain performance metricsexperienced at an endpoint device 116. Such indicia may, in someinstances, indicates buffering time, streaming disruptions, and/orpresentation degradation experience by the endpoint device 116. Suchindicia from an endpoint device 116 may be added to other data utilizedby the subsystem 111, which data may be utilized as bases for adjustmentof content composite 180 creation and/or the adaptation in order tooptimize the content composites 180 based at least in part on the data,including band with indicia. The optimization of the creation and/oradaptation may include selecting differing content items 177 contingenton the bandwidth constraints informed by the data. The type, size,and/or number of content items 177 may be selected based at least inpart on such data. By way of example, the subsystem 111 may createand/or adapt content composites 180 so that the content items 177, inaddition to the other components of the composites 180 meet bandwidthconstraints. Audio adaptation may utilize less bandwidth than videoadaptation, and the subsystem 111 may bias the adaptation toward audioadaptation over video adaptation in order to meet bandwidth constraints.Such biasing may include eliminating video adaptation and only employingaudio adaptation. Such biasing may include limiting video adaptation(e.g., only including one video content item 177 per composite 180), butallowing for more audio adaptation (e.g., including multiple audiocontent items 177 per composite 180). In certain instances, for example,the particularization may be constrained as a function of bandwidthspecifications such that the particularization may be limited so thatthe language variants of text is less than the language variants ofaudio (e.g., text in a single language, with audio in 12 differentlanguages). Further, as disclosed further herein, content items 176 maybe adapted with multiple content items 177 in various embodiments, andthe content composites 180 may be transmitted to the endpoint device 116with multiple content items 177, from which the endpoint device 116 mayselect for ultimate adaptation and presentation at the endpoint device116.

FIG. 3 is a simplified illustration of a composite build engine 240, inaccordance with disclosed embodiments of the present disclosure. Invarious embodiments, the composite build engine 240 may be included inthe subsystem 111 or may be separate from the subsystem 111. Thecomposite build engine 240 may, in some embodiments, be included in thecontent provider system 102. Having processed an adaptable content item176, the subsystem 111 may create one or more content composites 180that may include the adaptable content item 176. In various embodiments,the composites 180 created by the composite build engine 240 may includefully adapted content items 176 adapted with content items 177 fordelivery to endpoint devices 116, and/or content items 176 that are tobe adapted with content items 177 (e.g., content items 177 included inthe composites 180) after delivery to endpoint devices 116.

To create the content composites 180, disclosed embodiments may providethe content item 176 with containerization objects 182 that eachcontainerize a set of data as a persistent object in time. Eachcontainerization object 182 may be configured to function in a broadcastenvironment while facilitating the various features disclosed herein.For example, among other things, the containerization object 182 maycreate the ability to integrate deeply with endpoint devices 116 in abroadcast environment. With the containerization object 182, the contentcomposites 180 may allow the content item 176 to be treated as an objectwrapped with an identity and with one or more layers of encryption. Asdisclosed herein, the one or more layers of encryption provided withvarious embodiments may correspond to encryption at one or more softwarelayers, as distinguishable from hardware encryption of a device 116. Thevarious embodiments of one or more layers of encryption may allow forencryption of one or a combination of return channel communications,data regarding endpoint device 116 operations that contribute toobservation data 229, content items 176, content items 177, composites180, and/or components thereof, which may particularly includeencryption of personalization and transactional data, which may providefor DRM, and which may ensure end user privacy.

The composite build engine 240 may configure the containerization object182 to include a container flag 184. The container flag 184 may includeone or more parameters that indicate one or more containerizationspecifications 186. The container flag 184 may be thin and may indicateother components packetized in the containerization object 182. Forexample, in various embodiments, container flag 184 may include indiciaof one or a combination of: an identifier of the content item 176 in thecontainer, boundaries of the set of one or more audio and/or videopackets that form the content item 176, access and reference to one ormore other containerization specifications 186, access and reference toother metadata 188, and/or the like. The content item identifier maycorrespond to a field with parameters that identifies the content item176. Further, the composite build engine 240 may configure thecontainerization object 182 to include the containerizationspecifications 186 to facilitate performance of a set of one or moreoperations by the one or more endpoint media devices 116 with respect tothe content item 176 consequent to the one or more endpoint devices 116receiving the content composite 180. In various embodiments, thecontainerization specifications 186 may include one or a combination ofinstructions, metadata, personalization content or instructions to fetchpersonalization content, and/or the like to specify and facilitateperformance of the set of one or more operations by an endpoint mediadevice 116. In some embodiments, the container flag 184 may includeindicia of one or more content items 177 of the composite 180 that mayinclude one or a combination of: identification/referencing of thecontent items 177, and specifications for the configuration andadaptation with the content items 177 including any one or a combinationof the audio and/or video configuration aspects disclosed herein. Insome embodiments, the containerization specifications 186 may includeinstructions for configuring the composite 180 and content item 176 withone or more content items 177. In some embodiments, the containerizationspecifications 186 may include one or more particularized content items177 or instructions to fetch one or more content items 177.

Accordingly, the content composite 180 may include the content item 176and the containerization object 182. Further, in various embodiments,the content composite 180 may include one or more particularized contentitems 177 at the time the content composite 180 is transmitted to one ormore endpoint media devices 116, separate from or grafted into thecontent item 176 such that the content item 176 is an adapted contentitem 176 adapted with the one or more particularized content items 177.In some embodiments, the content composite 180 may not include aparticularized content item 177 at the time the content composite 180 istransmitted to one or more endpoint media devices 116. In suchinstances, the particularized content item 177 may be fetched per thecontainerization specifications 186 from the system 102, 200, anotherdata source 103, or from storage of the endpoint media device 116, andmay be used by an endpoint media device 116 to adapt the content item176. The containerization specifications 186 may include instructionsand build specifications according to which the endpoint media device116 may merge the pulled/retrieved particularized content item(s) 177with the content item 176. As part of the instructions and buildspecifications, the containerization specifications 186 may specify thatthe particularized content item 177 (e.g., a particular redeemablecontent item, promo code, matrix code such as a QR code, trailer, moviecontent, and/or the like) contain one or more date and/or timeattributes (e.g., a time stamp) indicate of one or a combination of whenthe item was created, when the item is to be output by the endpointmedia device 116 for display, when an interface element at the endpointmedia device 116 corresponding to the item is selected, when the item isstored by the endpoint media device 116 (e.g., in a digital wallet),and/or the like. In some embodiments, the one or more date and/or timeattributes may be stored with the metadata 188.

In various embodiments, the content composite 180 may further includepersonalization content 177. The personalization content 177 may beaudio, video, and/or textual content personalized to the end user of theendpoint media device 116 at the time the content composite 180 istransmitted to one or more endpoint media devices 116, separate from orgrafted into the content item 176 such that the content item 176 is anadapted content item 176 adapted with the personalization content. Insome embodiments, the content composite 180 may not includepersonalization content at the time the content composite 180 istransmitted to one or more endpoint media devices 116. In suchinstances, the personalization content may be fetched per thecontainerization specifications 186 from the system 102, 200, anotherdata source 103, or from storage of the endpoint media device 116, andmay be used by an endpoint media device 116 to adapt the content item176 according to instructions and build specifications of thecontainerization specifications 186. In some embodiments, the fetchedpersonalization content may be in one form and may be converted by theendpoint media device 116 into a different form. For example, textualcontent may be extracted from the retrieved personalization content(e.g., a list of items to acquire with a shopping list) in a file on themedia device 116, and the textual content may be reformatted and/ortranscribed to audio for inclusion in the composite 180 and presentationin the audio form along with the video and other audio of the composite180.

In some embodiments, the subsystem 111 may transmit the contentcomposite 180 to one or more endpoint media devices 116, and, in someembodiments, may broadcast the content composite 180 to a plurality ofendpoint media devices 116. The subsystem 111 may transmit the contentcomposite 180 in accordance with composite transmission specifications.The composite transmission specifications may at least partially specifytransmission parameters that govern transmission of the containerizationobject 182 in a data stream with the set of one or more audio and/orvideo packets in audio and/or video streams corresponding to the contentitem 176.

The content item 176 may correspond to a specific unit within thecontent stream. The specific unit in the content stream may correspondto a section of the video stream that includes the set of one or moreaudio and/or video packets for the content item 176, which may or maynot include one or more content item 177 in various embodiments (wherethe content item 176 may be already adapted with the item(s) 177 or maybe sent with a selected set of item(s) 177 for later adaptation with oneor more of the set). The composite transmission specifications mayspecify how the containerization object 182 should be transmitted in thedata stream with the content item 176, how many instances of thecontainerization object 182 should be created and transmitted, how someinstances of the containerization object 182 should be time/placeshifted in advance of the packets corresponding to the content item 176,and/or the like. Thus, the composite transmission specifications mayprovide for one or more instances of the containerization object 182particularized for the individual content item 176 to be transmittedwithin the data stream of the content stream, with the content item 176transmitted in the audio and/or video stream(s). Accordingly, a numberof instances of the containerization object 182 with the content itemidentifier, as specified by the composite transmission specifications,may be embedded in a section of the data stream that indicates thecontent item 176. Not only may the containerization object 182 with thecontainer flag 184 be carried as a data stream with the audiovideocontent stream, flagging the content item 176, it may also be replicatedand encoded in a carousel manner per the composite transmissionspecifications to facilitate pick-up mid-stream of the content item 176,and/or in advance of the content item 176 in the stream to facilitateforward identification of the upcoming content item 176. The containerflag 184 and its indicia may be encrypted and transmitted to theendpoint media device(s) 116 in the encrypted state with the contentcomposite 180. Additionally or alternatively, the containerizationspecifications 186 may be encrypted and transmitted to the endpointmedia device(s) 116 in the encrypted state with the content composite180. Accordingly, disclosed embodiments may add an encrypted containerflag 184 to the content item 176.

In various embodiments, the encrypted container flag 184 may governaccess by the endpoint media device 116 to one or more other componentsof the content composite 180, including the personalization content.Consequent to delivery of the content composite 180 to the endpointmedia device 116, the endpoint media device 116 may decrypt theencrypted container flag 184. Once the endpoint media device 116decrypts the container flag 184, the endpoint media device 116 may haveaccess to, for example, the metadata of the content composite 180, suchas the instructions of the containerization specifications 186, othercontainerization specifications 186, and/or the other metadata 188. Thecontainer flag 184 may include indicia that identifies the state andextent of one or more layers of encryption of the content composite 180.One or more of the components of the content composite 180 may beencrypted. The encryption may include a lightweight, point-to-pointencryption for specific components of the content composite 180, whichmay be advantageous to avoid bogging down processing in view ofrelatively heavier processing needs for video of content items 176 andgenerally shorter durations of content items 176.

The indicia may flag one or more objects and/or containers for thespecific unit(s) within the content stream without the entire datastream being encrypted. The subsystem 111 may all or part of contentcomposite 180 contingent on the extent of the personalization of thecontent composite 180. For example, when the content composite 180 istransmitted with personalization content, the personalization contentmay be encrypted. As one example out of many possibilities, when acontent item 176 is adapted with a list (in graphical and/or audio form)of items to acquire prepared for an identified viewer, at least theportion of data corresponding to the list may be encrypted, whereas thebalance of the content item 176 that is not personalized may notencrypted. Accordingly, a partially customized content composite 180and/or content item 176 may only be partially encrypted. However, ahighly customized content composite 180 and/or content item 176 may beentirely or substantially entirely encrypted, commensurate with theextent of the customization for the identified viewer. For example, ifthe content item 176 is due to an identified viewer selection (say, aprior procurement, such as the viewer having obtained access permissionsto a movie or other particular content), then, after scraping an ID thata viewer has allowed such, the content composite 180 and/or content item176 (which, for example, may be directed to opening up 1-to-1 connectionto access a digital ticket, access to a digital copy of the content,bonus content, a content item that requires tracking such as aredeemable content item for a pay-per-view, etc.) may be entirely orsubstantially entirely encrypted. Such content may be highly customizedand may include watermarking. One or more particularized content items177, for example, may include watermarks for merging with content items176 as disclosed herein.

The content composites 180 may provide for advantages with greaterlayers within a content stream more based around computer language. Insome embodiments, various options for content items 176, 177 may bebroadcasted, with only a subset of the content items 176, 177 beingpresented by particular endpoint media devices 116. The indicia of thecontainer flags 184 of the content composites 180 may allow forfiltering the content composites 180 according to attributes of theendpoint media device 116 and/or the endpoint viewer so the matchingcomposite 180 is presented to the identified viewer/viewer type,pursuant to the differentiated interaction with mobile app, etc. Forexample, flag indicia may include layer flag indicia that indicates thiscontent item is for unlocked devices or only locked devices. One layermay allow for identifying different types of the endpoint media devices116 that differ in capabilities, which, in some embodiments, may bedetermining based at least in part on pulling information (e.g., devicespecifications) from the particular endpoint media device 116. Suchdevice type identification may further allow for selectively providing atype of endpoint media device 116 with different types of content items176, 177 and different content attributes as a function of the devicetype. For example, such selective provisioning may include filteringcontent items 176, 177 based at least in part on the differences inframe rate of the content items 176, 177 as a function of the endpointmedia device type and/or capabilities. Different content items 176, 177may be filtered for different endpoint media device 116 (e.g., contentitems 176, 177 with frame rates associated with fast-changing contentmay not be mapped to devices with lesser capabilities). As anotherexample, such selective provisioning may include filtering content items176, 177 based at least in part on the differences in how the contentitems 176, 177 are compressed as a function of the endpoint media devicetype and/or capabilities.

One layer may allow for filtering one or a combination of the contentcomposite 180, the content item 176, the particularized content item177, and/or the personalization content based at least in part on theauthentication and identification of the endpoint viewer, e.g., based atleast in part on observation data. For example, such filtering mayinclude selecting and presenting the content composite 180, the contentitem 176, the particularized content item 177, and/or thepersonalization content that correspond to one or more demographics ofthe identified viewer (e.g., age, marital status, children, pets,renter, homeowner, etc.) and/or viewer type (e.g., flag indicia ofNC-17, MA, or another rating so an endpoint device with parentalspecifications would be provided with appropriate content items). Asanother example, such filtering may include selecting and presenting thecontent composite 180, the content item 176, the particularized contentitem 177, and/or the personalization content that correspond toidentified past actions mapped to the endpoint and/or endpoint mediadevice 116. This can result in such differences as presenting contentthat is directed to, say, “get video subscription service X” versus“because you have video subscription service X, . . . Y.” Another layermay allow for creating a backchannel to facilitate communications,ordering, secure video delivery, and/or the like disclosed herein.

In some embodiments, the container flag 184 and/or containerizationspecifications 186 may prompt the endpoint media device 116 to executethe instructions to perform at least one operation of the set of one ormore operations facilitated by the containerization object 182. In someembodiments, APIs may be used to instruct the endpoint media device 116as to what to do with the container flag 184 and/or containerizationspecifications 186. In some embodiments, the container flag 184 and/orcontainerization specifications 186 may allow for invoking, waking up,opening, and/or otherwise activating an application of the endpointmedia device 116 responsive to the decryption of the container flag 184and/or containerization specifications 186, in some instances, when theapplication is offline with respect to the system 102, 200, and/oranother system 103. This may include activating an application on anendpoint device 116 and prompting the saving of the ecoupon or promocode (which may be in audio form) in the endpoint device 116. Suchprompting may be performed at least in part by audio cues and commandsdirected to a virtual assistant of a nearby endpoint device 116 (e.g., asmart speaker) with audio indicia of the item.

For example, the container flag 184 and/or containerizationspecifications 186 may include a read flag that triggers one or morereturn channel communications. The corresponding instructions mayinstruct the endpoint media device 116 to report to the system 102, 200,and/or another system 103 with one or more return channel communicationsone or a combination of: that the content item 176 adapted with one ormore content items 177 was received by the endpoint media device 116;that the adapted content item 176 was presented with a display device atthe endpoint media device 116; to what extent the adapted content item176 was presented (in full or to what lesser extent in percentage oranother suitable metric); that the adapted content item 176 was shown toan intended endpoint user, detection of one or more operations executedconsequent to the presentation (e.g., audio transcription of a code orecoupon to an application of the media device 116 and/or another smartdevice, scanning a matrix code, loading a redeemable content item and/orparticularization content into a mobile app such as a digital wallet ofthe endpoint media device 116, utilizing the particular content items177 such as to order a service, etc.) and metrics of the operations(e.g., time of execution); and/or the like. Further, the return channelcommunications may include one or a combination of: indicia of aparticular endpoint user and/or type of endpoint user using the endpointmedia device 116 within a time period before and/or when at least partof an adapted content item 176 is/was output for display; indicia ofendpoint media device 116 capabilities (e.g., device specifications);indicia of authentication bases such as a SIM card of a mobile device,television receiver with a smartcard, and/or similar identification thatcan be used to authenticate to set up account; and/or the like.

The return channel communications may contribute to the observation data229 and repeated, iterative feedback loop features disclosed furtherherein. The containerization specifications 186 may instruct theendpoint media device 116 to report back upon occurrence of each statechange and/or to report back per a reporting schedule (e.g., just priorto closure of an application, at end of day, at the end of anotherreporting period, etc.). In various embodiments, the container flag 184and/or containerization specifications 186 may include one or moreencrypted messages, hashes, and/or the like that each indicate one ormore of the above reporting events. Accordingly, the instructions mayspecify that each return channel notification include a correspondingencrypted message, hash, code, and/or the like, which the system 102,200, and/or another system 103 may process to verify the respectivereporting event and metrics thereof.

In instances where the system 102 may receive pushed and/or pullednotifications from one or more service provider systems 103 thatindicate when particularized content items 177 have been used by theendpoint devices 116 with respect to the systems 103 (e.g., to a system103, uploading at least a portion of a content item 177; communicatingat least a portion of a content item 177 such as code, accessing using aURL of a content item 177; transferring, presenting, or otherwiseproviding an image capture of a content item 177; and/or the like), thenotifications may contain indicia of one or a combination of the contentitem 177, the endpoint device 116, the user associated with the endpointdevice 116, a location associated with the use of the content item 177,date and/or time associated with the use of the content item 177, and/orthe like. Having received such indicia from a service provider system103, the subsystem 111 (e.g., the matching engine 238) may correlate theindicia to records stored in a database 222 with identifiers of one or acombination of the content item 177, the endpoint device 116, the userand/or account associated with the endpoint device 116, a locationassociated with the user, account, and/or endpoint device 116, and/ordate and/or time associated with the delivery and/or presentation of thecontent item 177. The identified correlations may contribute to theobservation data 229 and feedback loop features.

In some embodiments, where the content item 176 is not already adaptedwith the particularized content item 177, the set of one or moreoperations may include identifying the particularized content item 177and adapting the content item 176 with the particularized content item177. When the particularized content item 177 is not included in thecontent composite 180 when the content composite 180 is received by theendpoint media device 116, the set of one or more operations may includepulling the particularized content item 177 from the system 102, 200,another data source 103 per the containerization specifications 186. Insome embodiments, the particularized content item(s) 177 may betransmitted separately from the system 102, 200 and/or another datasource 103 prior to the delivery of the content composite 180 such thatthe endpoint media device 116 stores the particularized content item(s)177 locally. In such instances, the set of one or more operations mayinclude retrieving the matching particularized content item 177 fromstorage of the endpoint media device 116 per the containerizationspecifications 186. Further, in some embodiments, when theparticularized content item 177 is not transmitted with the content item176, the containerization specifications 186 may specify first checkinglocal storage of the endpoint media device 116 for the matchingparticularized content item 177 and only pulling the particularizedcontent item 177 from the system 102, 200 and/or another data source 103when the particularized content item 177 is not available from the localstorage.

Similarly, the set of one or more operations may include identifying thepersonalized content 177 that is particular to one or more viewers andadapting the content item 176 with the personalized content 177. Asdisclosed herein, in some embodiments, the particularized content item177 may include personalized content 177. In such instances, theadapting of the content item 176 with the particularized content item177 may include adapting of the content item 176 with personalizedcontent 177. Additionally or alternatively, personalized content may beseparate from the particularized content item 177. Hence, where thecontent item 176 is not already adapted with the personalized content,the personalized content may be included in the content composite 180when the content composite 180 is received by the endpoint media device116. When the personalized content is not included in the contentcomposite 180 when the content composite 180 is received by the endpointmedia device 116, the set of one or more operations may includeretrieving/pulling the personalized content from one or a combination oflocal storage of the endpoint media device 116, the system 102, 200,and/or another data source 103 per the containerization specifications186. The containerization specifications 186 may specify first checkinglocal storage of the endpoint media device 116 for the matchingpersonalized content and then pulling the personalized content from thesystem 102, 200 and/or another data source 103 when at least some of thepersonalized content is not available (or when sufficient personalizedcontent of one or more specified types is not available) from the localstorage or when supplemental personalized content is identified asrequested per the containerization specifications 186.

To facilitate the content composite 180, the composite build engine 240may include a metadata handler 208 that may generate metadata (e.g., oneor more tags) corresponding to identifiers, attributes, characteristics,and/or categories of programming content 202, adaptable content items176, and/or particularized content items 177. In some embodiments, thecomposite build engine 240 may assign packet identifiers to identifydata of the content that is to be transmitted as part of a data streamto an endpoint media device 116 and that is to be associated with one ormore tags. Data transmitted as part of a data stream to an endpointmedia device 116 may be packetized and assigned packet identifiers. Aparticular packet identifier may be associated with data packets relatedto tags for particular segments of programming content 202, adaptablecontent items 176, and/or particularized content items 177.

The content matching engine 238 may identify particular content in theprogramming content 202 and may identify one or more correspondingidentifiers, attributes, characteristics, and/or categories ofprogramming content 202, adaptable content items 176, and/orparticularized content items 177 of one or more adaptable content items176 and/or one or more particularized content items 177. Based at leastin part on such identification, the composite build engine 240 maycreate metadata, which, in some embodiments, may correspond to tag data.Tag data may include an indication of a period of time (or other measureof time, e.g., a number of frames), a start frame, an end frame, and/orthe like. Tag data may include or otherwise be associated with a tagidentifier and may include event, attribute, characteristic, and/orcategory identifiers. For example, the metadata for the particularcontent may identify the particular content (e.g., show, movie, etc.).The metadata may further identify one or more attributes of theparticular event (e.g., any suitable identifier for the participatingentities, the location of an event, and/or the like). In someembodiments, at least a portion of the metadata augmentation may beperformed at the content provider system 102 such that one or moretagged composite components may be is provided to an endpoint mediadevice 116. Subsequently, the endpoint media device 116 may identifycomposite components, for example, by processing the metadata.

The metadata for adaptable content items 176 may, for example, identifythe adaptable content items 176 as being adaptable with any suitableidentifier, such as a flag, field value, etc. Additionally oralternatively, the metadata for the adaptable content items 176 and/or177 may identify with any suitable identifier that the adaptable contentitems 176 and/or 177 are designated for a certain event/programmingcontent, geolocation, category of events/programming content, types ofcontent (e.g., ratings for different audience designations, contentdesignations, etc.), types of programs and/or viewers, language/dialect,and/or the like. The metadata for the adaptable content items 176 and/or177 may further identify one or more attributes of the particular event(e.g., any suitable identifier for associated entities, location, atemporal attribute such as a time of an event, and/or the like).Additionally or alternatively, the metadata for the adaptable contentitems 176 may identify transition points, hooks, frames, windows, otherportions designated for overlays, and/or the like for merging withcontent items 177 such that content from the content items 177 is mergedat the transition points, hooks, frames, windows, other portionsdesignated for overlays, and/or the like.

In some embodiments, metadata-augmented adaptable content items 176 maybe provided by the service provider system 102 to the endpoint mediadevices 116, after which the endpoint media devices 116, each having atleast a portion of the content composite subsystems 111, may process anduse the metadata to facilitate matching adaptable content items 176 withcorresponding programming content 202. Likewise, the endpoint mediadevices 116 may process and use the metadata to facilitate matchingadaptable content items 176 with corresponding content items 177. Thus,the metadata may facilitate the endpoint media devices 116 appropriatelyproviding content composites 180 corresponding to certain contextualfactors for display with appropriate placement with respect to televisedevents, shows, etc. at commercial breaks and/or during presentation ofthe televised events, shows, etc.

In a similar manner, metadata for particularized content items 177 may,for example, identify a corresponding show, geolocations, categories ofevents/programming content, types of content (e.g., ratings fordifferent audience designations, content designations, etc.), types ofprograms and/or viewers, languages/dialects, and/or the like. Themetadata for the content items 177 may further identify fields andcontent for one or more attributes, such as phone numbers, emailaddresses, addresses, descriptors, URLs and hyperlinks to platforms andwebsites and/or sites for further information, ordering, and/or thelike. In some embodiments, at least a portion of the metadataaugmentation may be performed at the service provider system 102 and/orthe service provider system 103 with respect to the particularizedcontent items 177. Subsequently, the content composite subsystem(s) 111and/or endpoint media devices 116 may process and use the metadata tofacilitate matching adaptable content items 176 with correspondingcontent items 177. In some embodiments, the devices 116, having at leasta portion of the content composite subsystem 111, may process theparticularized content items 177 in the form in which they are received(e.g., directly from a service provider 103) and, based on suchprocessing, may match content items 176 with corresponding content items177.

Referring again more particularly to FIG. 2, the matching engine 238 maybe configured to match adaptable content items 176 to segments ofprogramming content 202 based at least in part on metadata at a serviceprovider system 102 side or at a device 116, in accordance with variousembodiments. For example, metadata may be extracted when or before agiven segment of programming content 202 is to be output for display andbefore a transition point. In some embodiments, the matching engine 238may read the metadata mapped to the segment and perform a search of therepositories 222 for one or more adaptable content items 176 that havemetadata matching the extracted metadata with respect to one or more ofevent identification, event category identification, and/or temporalidentification, with the highest preference given to the adaptablecontent item 176 that has metadata most closely matching the metadata ofthe previous segment. Alternatively, the matching engine 238 may readthe metadata mapped to the segment and pull one or more adaptablecontent items 176 from the service provider system 102. In so doing, thesubsystem 111 may transmit at least a portion of the metadata of theprogramming content 202 to the service provider system 102 in order tofacilitate matching the extracted metadata with one or more adaptablecontent items 176 with respect to one or more of event identification,event category identification, and/or temporal identification. Someembodiments may include the subsystem 111 configured to perform a searchof the repositories 222 for one or more adaptable content items 176 thathave metadata matching the extracted metadata in addition to pulling oneor more adaptable content items 176 from the service provider system102. For example, the subsystem 11I may first perform a search of therepositories 222 for any matching adaptable content items 176 and thenonly pull one or more adaptable content items 176 from the serviceprovider system 102 when no adaptable content items 176 are found in thesearch of the repositories 222 that match the extracted metadata with asufficient match score that indicates a level of correlation satisfyinga correlation threshold. Accordingly, certain embodiments may facilitatematching of adaptable content items 176 to segments of programmingcontent 202 based at least in part on metadata.

In a similar manner, the matching engine 238 may be configured to matchparticularized content items 177 to one or more of content items 176,segments of programming content 202, geolocations, categories ofevents/programming content, types of content (e.g., ratings fordifferent audience designations, content designations, etc.), types ofprograms and/or viewers, languages/dialects, and/or the like, inaccordance with various embodiments. For example, in some embodiments,the matching of one or more particularized content items 177 may bebased at least in part on metadata. The metadata may be extracted andread when or before a given content item 176 and/or a given segment ofprogramming content 202-1 is to be output for display, and, before atransition point, may be mapped to the content item 176 or the segment.In some embodiments, the matching of one or more particularized contentitems 177 may be based at least in part on identified characteristics ofa given content item 176, a given segment of programming content 202,geolocations, categories of events/programming content, types of content(e.g., ratings for different audience designations, contentdesignations, etc.), types of programs and/or viewers,languages/dialects, and/or the like. The matching engine 238 may takethe identified characteristics as cues to retrieve one or more contentitems 177 and/or one or more corresponding adaptable content items 176and for the content splicing engine 242 to output one or more contentcomposites 180 for presentation. The timing of the presentation may beafter the segment of programming content 202 (e.g., at a commercialbreak), during the segment of programming content 202 (e.g., as anoverlay, in a side window/frame, etc.) to match the identifiedcharacteristics of the programming content 202, or after thepresentation of a first composite 180 (e.g., when one or more contentitems 177 and/or one or more corresponding adaptable content items 176are selected for presentation as a subsequent composite 180 or forseparate provisioning to the device 116 in accordance with variousembodiments disclosed herein such as transferring to an accountaccessible by a user of the endpoint media device 116). In variousembodiments, the characteristic identification may include one or acombination of text recognition, image recognition, and/or audiorecognition. For example, the matching engine 238 may recognize text(which may include language and dialect) from the metadata of contentitems 176 and/or programming content 202. As some possible examples outof many possibilities, the matching engine 238 may recognize indicia ofuniform resource identifiers (e.g., URLs), identifiers of providersystems 103, and/or the like from the metadata (which may include closedcaptioning information) that the matching engine 238 may map to providersystems 103.

To facilitate image recognition, some embodiments of the contentcomposite subsystem 111 (e.g., by way of the matching engine 238) may beconfigured to detect one or more color characteristics in content items176 and/or programming content 202, which may include any suitable colorcharacteristic that may be measured to recognize alpha and/or numericalportions of the images and/or objects in the images. From such portionsand/or objects, the matching engine 238 may recognize uniform resourceidentifiers (e.g., URLs, web addresses, etc.), phone numbers,identifiers of provider systems 103, logos, graphics displayed withstreaming tickers or overlays typically positioned and/or scrollingacross a portion of the display area, and/or the like which the matchingengine 238 may map to provider systems 103. The matching engine 238 mayinclude an image analyzer and handling module to facilitate thatdetection. By way of example, the matching engine 238 may detect one ormore color characteristics in programming content 202 by way of any oneor combination of pixel identification, detecting color componentvalues, detecting color codes, and/or the like. Some embodiments maysample only portions of images, such as only part of a frame (whichcould, for example, be specified by identification of pixel coordinatesand ranges thereof to define areas of interest). Some embodiments maysample only a central portion of a frame. Other embodiments may onlysample a non-central portion, such as a border portion of a frame. Someembodiments may start with one portion of a frame and only sample tountil a recognition is achieved.

To facilitate audio recognition, the content composite subsystem 111(e.g., byway of the matching engine 238) may be configured to detect oneor more audio characteristics in content items 176 and/or programmingcontent 202. The matching engine 238 may include an audio analyzer andhandling module to facilitate that detection. By way of example, thematching engine 238 may detect one or more audio characteristics (whichmay include language and dialect) in content items 176 and/orprogramming content 202 by way of any one or combination of analyzingaudio, applying voice recognition, acoustic spectrum analysis, analyzingthe audio tracks metadata track, comparison to acoustic profiles fortypes of content, and/or the like. Some embodiments of the contentcomposite subsystem 111 may store acoustic profiles mapped to types ofcontent items 176 and/or programming content 202 and correspondingprovider systems 103 in the repository 222. Each acoustic profile mayspecify one or more selected values of acoustic metrics as distinctivemarkings and/or acoustic categories 214 (e.g., language, dialect, music,scores, jingles, voice characteristics, and/or the like) ascharacteristics of the types of content items 176 and/or programmingcontent 202 and corresponding provider systems 103.

As another example, the subsystem 111 may recognize one or more keywordsfrom dialogue that the matching engine 238 may map to language anddialect. The matching engine 238 may identify one or more keywordsand/or expressions in the dialogue as a dialogue impression for thepurposes of characterizing the sample and identifying content items 176and/or 177. The matching engine 238 may compile the keywords and/orexpressions of the sample and retain the keywords and/or expressions andcorresponding content items 176 and/or 177. In some cases, the matchingengine 238 may correlate the dialogue impression to one or more dialoguecategories 212 for similar dialogue impressions and correspondingcontent items 176 and/or 177. In various embodiments, the correlationmay be based at least in part on matching selected keywords and/orexpressions to identical and/or similar keywords and/or expressionsspecified for certain dialogue categories 212 and corresponding contentitems 176 and/or 177. The dialogue categories 212 may includecategorizations of concept, keyword, expression, and/or the like mappedto corresponding content items 176 and/or 177. Based at least in part onthe dialogue impression, the matching engine 238 may create a dialogueprofile for programming content 202 and corresponding content items 176and/or 177. The dialogue profile may be retained in any suitable form,such as a file, a list, etc. The content composite subsystem 111 maystore dialogue profiles mapped to content items 176 and/or 177 and/orprogramming content 202 in the repository 222. Each dialogue profile mayspecify one or more selected keywords and/or expressions as distinctivemarkings and/or dialogue categories 212 as characteristics of thecontent items 176 and/or 177 and/or programming content 202.

To facilitate the matching, the matching engine 238 may performcorrelation based at least in part on the text recognition, imagerecognition, and/or audio recognition and text, image, and/or audiocategories 212. By way of example, the correlation rules 218 may includecorrelation criteria that could include keywords identified by any oneor combination of words, word stems, phrase, word groupings, and/or likekeyword information. The correlation criteria could include weightingsassigned to dialogue specifications. Hence, within each dialogue bucket,a keyword could be assigned a weight according to its significance. Moresignificant keywords could be assigned a greater weight than stop words.Such buckets could be implemented in any suitable manner, includinglists, tables, matrices, and/or the like. And within a bucket,characteristics and/or keywords could be organized in a rank orderand/or any hierarchical structure according to weight. Some embodimentsmay have characteristics and/or keywords organized according to decisiontree, with contingencies so that only certain combinations ofcharacteristics and/or keywords may be considered. For example, certaincharacteristics and/or keywords could only have significance if used inconjunction with other characteristics and/or keywords, and/or not inconjunction with others. Similar correlation criteria may be used foracoustic impressions. In some embodiments, the matching engine 238 mayemploy a scoring system to quantify correlations with a numericalexpression, for example, a match score, with higher scores beingassigned to higher correlations. Higher scores may be assigned forgreater extents of matching. By way of example with respect to dialogue,a match of three dialogue cues or dialogue categories may be assigned ahigher score than a match of only one top dialogue cue and/or dialoguecategory. As another example, an exact match of words, expressions,and/or dialogue categories may be assigned a higher score than acomplimentary match (e.g., where absent an exact match, a word,expression, and/or dialogue category is defined as a complement toanother a word, expression, and/or dialogue category). Additionalaspects, features, and details relating to the subsystem 111, thecontent items 176, 177, and the composites 180 are provided inapplication Ser. No. 16/655,104, incorporated by reference herein.

Particularized content items 177 may be provisioned to endpoint mediadevices 116 in one or a combination of ways disclosed herein. Forexample, in some embodiments, one or more particularized content items177 may be integrated with and audially presented with the adaptablecontent item 176. Such presentations of the particularized content items177 may be captured by an endpoint media device 116 listening via amicrophone integrated with or other communicatively coupled to the mediadevice 116. Additionally or alternatively, particularized content items177 may be transferred to an account accessible by the user of theendpoint media device 116. In various embodiments, the particularizedcontent items 177 may be transferred to the account in audio form infull or only partially (e.g., after the subsystem 111 and/or mediadevice 116 extracts a pertinent option, such as a promo code, from thefull audio), and/or in graphical, textual, or the like form (e.g., afterthe subsystem 111 and/or media device 116 transcribes an audio segmentportion). Additionally or alternatively, particularized content items177 may be provisioned to a device 116 by way of one or combination ofapplication installed on the device 116, communications from a receiver116 e, communications from the service provider system 102, a near fieldcommunication interface (e.g., contactless interface, Bluetooth, opticalinterface, etc.), wireless communications interfaces capable ofcommunicating through a cellular data network, or through Wi-Fi, such aswith a wireless local area network (WLAN), and/or the network 120. Byway of example, a composite 180 may be presented via a televisionreceiver 116 e on a display 160 with one or more user-selectable optionsthat allow redirection of composites 180 and/or particularized contentitems 177 to a secondary device 116 consequent to user selection.Further, one or more user-selectable options may be presented to theuser via the receiver 116 e and/or device 116 to allow content composite180 and/or particularized content item 177 provisioning to a secondarydevice 116 concurrently with the presentation of the televised contentsegment. As disclosed herein, the user-selectable options may includeone or more options to transition provisioning of one or more composites180 to a secondary device 116 and/or to request notifications from theone or more platforms, websites, and/or sites be sent to the secondarydevice 116 so that the secondary device 116 may be used to interact withthe platforms and sites via an application installed on the secondarydevice 116. In that way, a user may interact with the one or moreplatforms and sites via the secondary device 116 while viewing thetelevised event on a primary display associated with a receiver 116 eand/or primary device 116.

As illustrated by FIG. 2, the content processing subsystem 111 mayinclude a learning engine 239 that may be an analysis engine thatemploys machine learning. The learning engine 239 may further employdeep learning. Accordingly, the learning engine 239 may facilitatemachine learning or, more specifically, deep learning, to facilitatecreation, development, and/or use of observation data including viewerpattern data 216. As disclosed herein, the subsystem 111 may determine aprogram, show, etc. that the viewer actually is viewing, is about toview (e.g., the televised event is specified to play on the channel thatthe viewer is currently viewing), or is likely to view as determinedwith the learning engine 239. The subsystem 111 may further determine atype of audiovisual content (e.g., ratings for different audiencedesignations, content designations, etc.) and intra-content metricscorresponding to indicia or specifications of languages and dialects ofdialogues and/or text within the content and/or selected closedcaptioning. The subsystem 111 may push information indicative of thecontent, content type, and/or metrics to one or more service providersystems 102 and/or 103. In some embodiments, the service provider system102 may select one or more content items 176, 177 matching the contentsegment for transfer to the subsystem 111 which, as disclosed herein,may be a part of the content provider system 102 and/or may be part ofthe receiver 116 e and/or devices 116. The subsystem 111 may select fromthe one or more adaptable content items 176, 177 as matching particularsegments of the televised segment and, utilizing the content splicingengine 242 in some embodiments, may output one or more correspondingcontent composites 180 for presentation after the particular segmentsand/or simultaneously with the particular segments.

In various embodiments, one or more of the service provider systems 102,103 may select one or more particularized content items 177 matching thetelevised event for transfer to the subsystem 111. In some embodiments,one or more of the service provider systems 102, 103 may select a set ofone or more particularized content items 177 for transfer (e.g., basedon recency of information updates corresponding to the content items177) for transfer to the subsystem 111, and the subsystem 111 maydetermine which content items 177 from the set match the televisedcontent segment. As disclosed above, the content composite subsystem 111may include a matching engine 238 that may include logic to implementand/or otherwise facilitate any taxonomy, classification,categorization, correlation, mapping, qualification, scoring,organization, and/or the like features disclosed herein.

FIG. 4 illustrates certain aspects of the artificial-intelligence-basedsubsystem data flow 400, in accordance with various embodiments of thepresent disclosure. The content processing subsystem 111 may beconfigured to gather observation data 229, which may be specific to oneor more particular identified users and/or may be generally related toparticular endpoint media devices 116. The observation data 229 may begathered from one or more devices 116, aggregated, consolidated, andtransformed into viewer pattern profiles that include personalizedpattern data 216.

In embodiments where the learning engine 239 is included in an endpointmedia device 116, the device 116 may be a self-observer that mayadditionally gather additional observation data 229. In variousembodiments, the data from the one or more devices 116 may be retrievedand/or received by the content processing subsystem 111 via one or moredata acquisition interfaces, which may include interfaces of the contentprocessing subsystem 111, the one or more endpoint media devices 116,and/or the like-through network(s) 120 in various embodiments, throughany suitable means for direct communication, and/or through any othersuitable means of transferring data. According to various embodimentswhere the subsystem 111 is included in a service provider system 102,observation data 229 may be actively gathered and/or pulled from the oneor more endpoint media devices 116. As disclosed herein, in variousembodiments, the one or more data acquisition interfaces may include oneor more APIs that define protocols and routines for interfacing with theone or more endpoint media devices 116 and which may specify API callsto/from one or more endpoint media devices 116. In various embodiments,the APIs may include a plug-in to integrate with an application of oneor more endpoint media devices 116. The API translation profiles maytranslate the protocols and routines of the data source component and/orsystem to integrate at least temporarily with the system and allowone-way communication to the system 102 and/or two-way communicationwith system 102 in various embodiments by way of API calls.

Some embodiments of the subsystem 111 may aggregate observation data 229to derive device identification data 404, device operations 406,temporal data 408, and/or contextual data 410. The device identificationdata 404 may include any suitable data for identifying and trackingparticular receivers 116 e and devices 116; associated accounts,subscribers, and viewers; and/or the like disclosed herein. The deviceoperations data 406 may include any suitable data for identifying andtracking device operations and interactions as those disclosed herein.The contextual data 410 may include metrics and patterns of viewerinteractions/responses pursuant to provisioning of content composites180 and service provider system 103 biasing 181. For example, viewerresponses to content composites 180 provisioning may include indicationsof whether the viewer selected a user-selectable options provided withcontent items 177 and composites 180, the types of such selections,and/or types of consequent interactions with service provider systems103. For example, the metrics and patterns may take into account whetherthe viewer opted out of content composites 180, whether the viewerselected links of composites 180 to interact with the platforms andsites of one or more service provider systems 103, whether the viewerselected options to redirect content composites 180 and/or notificationsfrom service provider systems 103 to a secondary device 116, whichoptions and/or service provider systems 103 the viewer selected, whetherthe viewer used redeemable content items, promo codes, QR codes, etc.and the types of the viewer's orders, purchases, and other interactionswith service provider systems 103, and/or the like. The temporal data408 may include metrics such as any information to facilitate detection,recognition, and differentiation of one or combination of temporalfactors correlated or which the content processing subsystem 111correlates to other observation data 229 such as device identificationdata 404, contextual data 410, and/or the like. For example, thetemporal data 408 may include time of day information, time of weekinformation, time of year information, holiday information, etc. whenthe viewer made selections, orders, purchases, redemptions of redeemablecontent items, offers, promo codes, etc.; and/or the like.

The learning engine 239 may map one or a combination of the variousextra-composite metrics of the observation data 229 to the metrics ofthe particular composites 180 provided to a particular viewer. Based atleast in part on taking into account such observation data 229 as partof an iterative feedback loop, the learning engine 239 may employ anongoing learning mode to develop personalized pattern data 216 forparticular viewers or content receivers/devices generally, and toconfirm, correct, and/or refine determinations made for personalizedpattern data 216 for particular viewers or content receivers/devicesgenerally. The content processing subsystem ill may be configured toemploy machine learning to process the observation data 229 and thecontent items 180 and to derive and develop the personalized patterndata 216. The content processing subsystem 111 may be configured toemploy deep learning to process the observation data 229 and the contentitems 180 and to derive and develop the personalized pattern data 216.The learning engine 239 may be configured to perform any one orcombination of features directed to matching or otherwise correlatingthe observation data 229—such as the device identification data 404, thedevice operation identification data 406, the temporal data 408, thecontextual data 410, descriptive information of the content items 180,and/or the like—with intra-content metrics of the content items 180. Thelearning engine 239 may include logic to implement and/or otherwisefacilitate any taxonomy, classification, categorization, correlation,mapping, qualification, scoring, organization, and/or the like featuresdisclosed herein. In some embodiments, the learning engine 239 mayinclude the matching engine 238. The learning engine 239 may include areasoning module to make logical inferences from a set of the detectedand differentiated data to infer one or more patterns of activity forparticular viewers and/or receivers/devices generally. A pattern-basedreasoner could be employed to use various statistical techniques inanalyzing the data in order to infer personalized pattern data 216 fromthe observation data 229. A transitive reasoner may be employed to inferrelationships from a set of relationships related to the observationdata 229. In various embodiments, the system automatically establishesand develops the personalized pattern data 216. However, thepersonalized pattern data 216 may be set up and/or tailored by users.With various embodiments, the personalized pattern data 216 may beautomatically established and developed by the system.

The feedback could be used for training the system to heuristicallyadapt conclusions, profiles, correlations, attributes, triggers,patterns, and/or the like to learn particular viewers and adapt contentcomposite 180 provisioning to particular viewers, which may includerequesting, searching for, and/or selecting particular types ofadaptable content items 176 and/or content items 177 (e.g., which may bebased at least in part on the metadata features disclosed herein) forcontent composite 180 creation and adaptation. For example, the learningengine 239 may learn that a particular viewer tends to interact withcertain types of content composites 180. Such interaction object typedifferentiation may be on the macro level, such as recognizing that aviewer tends to interact more with content composites 180 that aredirected to audio adaptions with audio content items 177 more than withcontent composites 180 that are directed to audio adaptions with audiocontent items 177. Further, viewer interaction differentiation may be onthe micro level, such as recognizing that a viewer tends to interactmore with composites 180 directed to content items 117 corresponding tocertain types of offers, redeemable content items, promotions, etc.Still further, the learning engine 239 may learn that a particularviewer tends to interact with content composites 180 that are directedto audio content items 177 in a certain language and dialect. Evenfurther, the learning engine 239 may learn that a particular viewertends to interact with content composites 180 where the content items177 are temporally positioned near the beginning of the respectivecontent items 176 for presentation primacy and/or with contentcomposites 180 where the content items 177 are temporally positionednear the ending of the respective content items 176 for presentationrecency. Accordingly, based at least in part on one or a combination ofsuch learnings for a particular viewer, the subsystem 111 may bias theselection, creation, and/or adaptation of content composites 180 towardthe types, configurations, and/or positioning of content items 177 thattend to elicit viewer interaction, and decrease or cease the selection,creation, and/or adaptation of content composites 180 toward othertypes, configurations, and/or positioning of content items 177.

The methods, systems, and devices discussed above are examples. Variousconfigurations may omit, substitute, or add various procedures orcomponents as appropriate. For instance, in alternative configurations,the methods may be performed in an order different from that described,and/or various stages may be added, omitted, and/or combined. Also,features described with respect to certain configurations may becombined in various other configurations. Different aspects and elementsof the configurations may be combined in a similar manner. Also,technology evolves and, thus, many of the elements are examples and donot limit the scope of the disclosure or claims.

Further details regarding embodiments of receivers corresponding toreceivers 116 e and of computing devices 116 that make use of, interactwith, and/or at least partially include the subsystem 111 are disclosedin the applications incorporated by reference herein. Likewise, furtherdetails regarding a computer system that may be incorporated as part ofthe previously described computerized devices, such as a televisionreceiver 116 e, device(s) 116, and/or content provider system 102 aredisclosed in the applications incorporated by reference herein. Specificdetails are given in the description to provide a thorough understandingof example configurations (including implementations). However,configurations may be practiced without these specific details. Forexample, well-known circuits, processes, algorithms, structures, andtechniques have been shown without unnecessary detail in order to avoidobscuring the configurations. This description provides exampleconfigurations only, and does not limit the scope, applicability, orconfigurations of the claims. Rather, the preceding description of theconfigurations will provide those skilled in the art with an enablingdescription for implementing described techniques and features. Variouschanges may be made in the function and arrangement of elements withoutdeparting from the spirit or scope of the disclosure.

What is claimed:
 1. A method for audio adaption of content items todevice operations of an endpoint media device, the method comprising:processing, by one or more processing devices, a first set ofobservation data corresponding to indications of detected media deviceoperations associated with a first media device and mapped to a firstset of content items; receiving, by the one or more processing devices,a first content composite comprising an adaptable content itemcorresponding to a set of one or more audio and/or video packetscorresponding to audio and/or video content, the first content compositefrom a content provider system; adapting, by the one or more processingdevices, the first content composite with a first audio segment, theadapting comprising: selecting, based at least in part on the first setof observation data, the first audio segment from a plurality of audiosegments; and configuring the first content composite with the firstaudio segment so that the adapted first content composite plays thefirst audio segment for at least part of a first presentation time whenthe adapted first content composite is presented; outputting, by the oneor more processing devices, the adapted first content composite forpresentation where, consequent to providing the adapted first contentcomposite to a first endpoint media device or a second endpoint mediadevice, the first endpoint media device or the second endpoint mediadevice performs at least one operation relating to the adapted firstcontent composite; processing, by the one or more processing devices, atleast one communication received from the first endpoint media device,the second endpoint media device, or a remote system that is remote fromthe one or more processing devices, the first endpoint media device, andthe second endpoint media device, the at least one communicationindicative of the at least one operation relating to the adapted firstcontent composite; adapting, by the one or more processing devices, asecond content composite with a second audio segment, the adaptingcomprising: selecting, based at least in part on the at least onecommunication indicative of the at least one operation relating to theadapted first content composite, the second audio segment; andconfiguring the second content composite with the second audio segmentso that the adapted second content composite plays the second audiosegment for at least part of a second presentation time when the adaptedsecond content composite is presented; and outputting, by the one ormore processing devices, the adapted second content composite forpresentation where, consequent to providing the adapted second contentcomposite to the first endpoint media device or the second endpointmedia device, the first endpoint media device or the second endpointmedia device performs at least one operation relating to the adaptedsecond content composite.
 2. The method for audio adaption of contentitems to device operations of an endpoint media device as recited inclaim 1, further comprising: receiving the plurality of audio segmentsfrom the content provider system.
 3. The method for audio adaption ofcontent items to device operations of an endpoint media device asrecited in claim 2, where the plurality of audio segments are receivedwith the first content composite.
 4. The method for audio adaption ofcontent items to device operations of an endpoint media device asrecited in claim 3, where the first content composite is adapted withthe first audio segment as a function of time.
 5. The method for audioadaption of content items to device operations of an endpoint mediadevice as recited in claim 4, where the first set of observation datacomprises a viewer metric as a function of time, and the first contentcomposite is adapted with the first audio segment as a function of theviewer metric.
 6. The method for audio adaption of content items todevice operations of an endpoint media device as recited in claim 5,where the first set of observation data comprises a current geolocationof the first endpoint media device, and the first content composite isadapted with the first audio segment as a function of the currentgeolocation.
 7. The method for audio adaption of content items to deviceoperations of an endpoint media device as recited in claim 6, where theat least one operation relating to the adapted first content compositecorresponds to storing a code corresponding to the first audio segmentby the first endpoint media device and/or the second endpoint mediadevice.
 8. A system to audio adapt content items to device operations ofan endpoint media device, the system comprising: one or more processingdevices; and memory communicatively coupled with and readable by the oneor more processing devices and having stored therein processor-readableinstructions which, when executed by the one or more processing devices,cause the one or more processing devices to perform operationscomprising: processing a first set of observation data corresponding toindications of detected media device operations associated with a firstmedia device and mapped to a first set of content items; processing afirst content composite comprising an adaptable content itemcorresponding to a set of one or more audio and/or video packetscorresponding to audio and/or video content, the first content compositefrom a content provider system; adapting the first content compositewith a first audio segment, the adapting comprising: selecting, based atleast in part on the first set of observation data, the first audiosegment from a plurality of audio segments; and configuring the firstcontent composite with the first audio segment so that the adapted firstcontent composite plays the first audio segment for at least part of afirst presentation time when the adapted first content composite ispresented: outputting the adapted first content composite forpresentation where, consequent to providing the adapted first contentcomposite to a first endpoint media device or a second endpoint mediadevice, the first endpoint media device or the second endpoint mediadevice performs at least one operation relating to the adapted firstcontent composite; processing at least one communication received fromthe first endpoint media device, the second endpoint media device, or aremote system that is remote from the one or more processing devices,the first endpoint media device, and the second endpoint media device,the at least one communication indicative of the at least one operationrelating to the adapted first content composite; adapting a secondcontent composite with a second audio segment, the adapting comprising:selecting, based at least in part on the at least one communicationindicative of the at least one operation relating to the adapted firstcontent composite, the second audio segment; and configuring the secondcontent composite with the second audio segment so that the adaptedsecond content composite plays the second audio segment for at leastpart of a second presentation time when the adapted second contentcomposite is presented; and outputting the adapted second contentcomposite for presentation where, consequent to providing the adaptedsecond content composite to the first endpoint media device or thesecond endpoint media device, the first endpoint media device or thesecond endpoint media device performs at least one operation relating tothe adapted second content composite.
 9. The system to audio adaptcontent items to device operations of an endpoint media device asrecited in claim 8, further comprising: receiving the plurality of audiosegments from the content provider system.
 10. The system to audio adaptcontent items to device operations of an endpoint media device asrecited in claim 9, where the plurality of audio segments are receivedwith the first content composite.
 11. The system to audio adapt contentitems to device operations of an endpoint media device as received inclaim 10, wherein the first content composite is adapted with the firstaudio segment as a function of time.
 12. The system to audio adaptcontent items to device operations of an endpoint media device asrecited in claim 11, where the first set of observation data comprises aviewer metric as a function of time, and the first content composite isadapted with the first audio segment as a function of the viewer metric.13. The system to audio adapt content items to device operations of anendpoint media device as recited in claim 12, where the first set ofobservation data comprises a current geolocation of the first endpointmedia device, and the first content composite is adapted with the firstaudio segment as a function of the current geolocation.
 14. The systemto audio adapt content items to device operations of an endpoint mediadevice as recited in claim 13, where the at least one operation relatingto the adapted first content composite corresponds to storing a codecorresponding to the first audio segment by the first endpoint mediadevice and/or the second endpoint media device.
 15. One or morenon-transitory, machine-readable media having machine-readableinstructions thereon which, when executed by one or more processingdevices, cause the one or more processing devices to perform operationscomprising: processing a first set of observation data corresponding toindications of detected media device operations associated with a firstmedia device and mapped to a first set of content items; processing afirst content composite comprising an adaptable content itemcorresponding to a set of one or more audio and/or video packetscorresponding to audio and/or video content, the first content compositefrom a content provider system; adapting the first content compositewith a first audio segment, the adapting comprising: selecting, based atleast in part on the first set of observation data, the first audiosegment from a plurality of audio segments; and configuring the firstcontent composite with the first audio segment so that the adapted firstcontent composite plays the first audio segment for at least part of afirst presentation time when the adapted first content composite ispresented; outputting the adapted first content composite forpresentation where, consequent to providing the adapted first contentcomposite to a first endpoint media device or a second endpoint mediadevice, the first endpoint media device or the second endpoint mediadevice performs at least one operation relating to the adapted firstcontent composite; processing at least one communication received fromthe first endpoint media device, the second endpoint media device, or aremote system that is remote from the one or more processing devices,the first endpoint media device, and the second endpoint media device,the at least one communication indicative of the at least one operationrelating to the adapted first content composite; adapting a secondcontent composite with a second audio segment, the adapting comprising:selecting, based at least in part on the at least one communicationindicative of the at least one operation relating to the adapted firstcontent composite, the second audio segment; and configuring the secondcontent composite with the second audio segment so that the adaptedsecond content composite plays the second audio segment for at leastpart of a second presentation time when the adapted second contentcomposite is presented; and outputting the adapted second contentcomposite for presentation where, consequent to providing the adaptedsecond content composite to the first endpoint media device or thesecond endpoint media device, the first endpoint media device or thesecond endpoint media device performs at least one operation relating tothe adapted second content composite.
 16. The one or morenon-transitory, machine-readable media as recited in claim 15, furthercomprising: receiving the plurality of audio segments from the contentprovider system.
 17. The one or more non-transitory, machine-readablemedia as recited in claim 16, where the plurality of audio segments arereceived with the first content composite.
 18. The one or morenon-transitory, machine-readable media as recited in claim 17, where thefirst content composite is adapted with the first audio segment as afunction of time.
 19. The one or more non-transitory, machine-readablemedia as recited in claim 18, where the first set of observation datacomprises a viewer metric as a function of time, and the first contentcomposite is adapted with the first audio segment as a function of theviewer metric.
 20. The one or more non-transitory, machine-readablemedia as recited in claim 19, where the first set of observation datacomprises a current geolocation of the first endpoint media device, andthe first content composite is adapted with the first audio segment as afunction of the current geolocation.